Memory management in event recording systems

ABSTRACT

A vehicle event recorder is provided that includes a camera for capturing a video as discrete image frames, and that further includes a managed loop memory and a management system for generating a virtual ‘timeline dilation’ effect. To overcome size limits in the buffer memory of the video event recorder, the maximum time extension of a video series is increased by enabling a reduction in temporal resolution in exchange for an increase in the temporal extension. Memory cells are overwritten in an ‘interleaved’ fashion to produce a reduced frame rate for the recording of certain time periods connected to an event moment. In time periods furthest from the event moment, the resulting frame rate is minimized while in time periods closest to the event moment, the resulting frame rate is maximized.

FIELD OF THE INVENTION

The following invention relates to electronic storage of data. Moreparticularly, the present invention relates to systems and methods formemory allocation and writing in devices recording vehicle events.

BACKGROUND OF THE INVENTION

Video recording systems are commonly used to monitor places where it maybe desirable to monitor security levels. For example, in a securityzone, breaches of security may be recorded if video monitors areaarranged around the space to be monitored. When an incident occurs, avideo record of the related activity is available from the recordingsystem.

During periods when no breaches of security occur, a considerable amountof data is still generated by the video recording system, which haslittle or no value. Thus, such data can be readily discarded withoutloss. The act of ‘discarding’ data amounts to merely rewriting new dataover old recorded data. Indeed, most video security systems are arrangedwith a recording medium that is reused continuously.

When a video camera generates an amount of image data sufficient tofully consume the available memory, newly collected data is recorded atthe beginning of the memory. Therefore, the act of writing newlyacquired data causes old data to be discarded; that is, the old data islost to the ‘write’ operation of the newly collected data.

In old videotape systems, this is sometimes called a ‘round robin’arrangement. A memory medium fashioned as a tape in a continuous loopprovides data storage for those video security systems. In such systemsthe tape has no end and no-beginning, but instead the tape continuouslypasses by a recording head where new images are written to the tape andat the same time old images are discarded.

When an incident of interest occurs, the tape may be stopped to preventloss of data which relates to the relevant incident. The related imagesmay be recovered from the tape and transferred to a permanent medium,while the tape is returned to the video system for further recording.Such re-use of memory is well known in the prior art.

In a round-robin scheme, the data that is overwritten, or discarded, isthe data which came into the system earliest, or was ‘first in’ thesystem. This arrangement is sometimes referred to as “first-in, firstoverwritten,” which is analogous to the “first-in, first-out”arrangement in electronic buffer systems. In both cases, we will referto this method as FIFO.

A FIFO base system is generally a very good system for buffermanagement, because the oldest data in a buffer is typically the leastvaluable. Therefore, the oldest data, or the least valuable data, may bediscarded without regard for its loss. However, in buffer systems theearliest received data, or the ‘first-in’ data, may not always be theleast valuable data. In some instances it may be advisable not tooverwrite the oldest data, but rather to provide an overwrite schemewhich preserves certain portions of interest of the oldest data.

With specific reference to the monitoring of vehicle activities, vehicleevent recorders are video recorder systems mounted within the vehicle toprovide a video record related to the environment surrounding a vehicleduring its operation. These systems are known in the art and areemployed, for example, in conjunction with police activity. Many policedepartments in the United States are equipped with vehicle eventrecorders, which capture activity, sometimes criminal, occurring in theproximity of a police vehicle.

The application of vehicle event recorders is not limited to policevehicles. More and more commercial vehicles are now equipped withsystems that record activity associated with the use of the vehicle andwithin the environments in which the vehicle is used. These systems areparticularly advantageous with fleet vehicles that are subject to heavyprofessional use and frequent incidents, including traffic accidents,theft, and vandalism, among others. With a video record, vehicle fleetmanagers are better equipped to manage and control costs associated withoperations of large vehicle fleets. Safety is improved, driverperformance is improved, a better understanding of accidents isachieved, and other benefits are derived from the use of vehiclerecorder systems.

Vehicle recorder systems are described, for example, in U.S. Pat. Nos.6,389,340, 6,405,112, 6,449,540, and 6,718,239, all to Rayner. Ingeneral, these inventions relate to a small device mounted on thevehicle rearview mirror to capture video images of traffic incidentsahead of the vehicle.

In particular, the '112 patent discloses a system that includes avehicle operator performance monitor, which records a video of thevehicle operator, and which may be used to determine how the operator'sactions affect use of the vehicle.

The '540 patent instead is an event recorder mounted in a vehicle, whichincludes one or more wave pattern detectors for detection andrecognition of a predetermined wave produced outside the vehicle, andfor producing a trigger signal denoting the presence of thepredetermined wave. In particular, a detective wave is a wave of thetype produced by a police or fire department emergency vehicle.Detection of this wave triggers a capture function which stores videoimages a long-term storage memory.

The '329 patent includes a video recorder system having a one-way hashfunction to perform a validation function. This way, the integrity ofthe recorded video data can be protected.

Finally, the '340 patent teaches a recording system having a certainrelationship between two different types of memory. A first memory isarranged to store video for the short-term and to transfer some of thatstored video in response to a trigger event. Data from this short termmemory is transferred to a more durable and long term memory, and theshort term memory is continuously overwritten in a scheme described byRayner as “first-in, first-overwritten”. This way, Rayner teaches thecoupling of a high-speed, high-performance volatile semiconductor memorywith a flashlight memory good for long-term storage of large amounts ofdata even when power is removed. As will be described in detail later,Rayner's first in first overwritten scheme necessarily creates a loss ofimportant and valuable data.

While systems and inventions in the prior art are designed to achieveparticular goals and objectives, these inventions also includelimitations which prevent their use in more extended applications, andcannot be used to realize the advantages and objectives of theinventions taught hereinafter.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide high-performancememory allocation systems for vehicle event recorders.

It also is an object of the present invention to provide for set memoryallowances and for extended recording periods.

It is a further object to provide memory overwrite schemes that providea time dilation on either end of a recording period.

These and other objects of the present invention are accomplished byapparatus and methods for allocating memory in vehicle event recordingsystems that provide improved memory allocation and writing schemes topreserve data over extended time periods, thereby improving prior artdevices and methods that cannot record events over a large time rangewhen equipped with a memory of a predetermined size.

One difference between memory allocation of the present invention andthe prior art can be seen in relation to time dilation, which may beapplied on the extremities of discrete capture periods. In oneembodiment, a memory of limited size is subject to an advanced managedloop memory allocation scheme and the rate of storage frame is adjustedthroughout a predetermined capture time period. At the extremities of atime period, a frame capture process is subject to a reduced frame rate.In the proximity to an event of interest, or an ‘event moment’, theframe rate-is increased to improve detail around that particular time,enabling a greater temporal range at the expense of temporal resolution.Losses in temporal resolution are however pushed away from the eventmoment and allocated to the extremities of the capture time period.

An overwrite scheme selects which frames are expired and subject tobeing discarded. At any random moment, video is continuously captured ata maximum frame rate. However, these frames are not stored into memoryin a conventional first-in, first overwritten manner, but rather, theseframes are added to the memory locations which are determined to beavailable in accordance with the overwrite scheme. Therefore, the oldestframes are not necessarily those that are overwritten. Quite to thecontrary, an overwrite action may be applied to a frame which is newerthan at least one other frame stored in the memory.

As a result, newly acquired frames are placed into memory in processeswhich necessarily cause older frames to be discarded. However, the newlycaptured frames are written to memory positions in an ‘interleaved’fashion, whereby some of older frames are preserved. When a captureevent occurs, data in the memory may be transferred to a more permanentstorage. When data is transferred, a timeline is reconstructed. Therecorded timeline is unique in that it contains various frame rates overthe capture period. At both the beginning and at the end of the captureperiod, the frame rate is modest. At the point of greatest interest inthe capture period, the frame rate is the highest. This throttling offrame rate provides for a memory of given size to accommodate a timelineof greater temporal extent.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects and advantages of the present inventions willbecome better understood upon consideration of the followingdescription, taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a timeline illustration in conjunction with a graphic thatillustrates a plurality of memory bin units;

FIG. 2 is a timeline illustration and memory graphic illustrating anoverwrite operation;

FIG. 3 illustrates the overwrite operation in conjunction with a triggerevent;

FIG. 4 illustrates particular memory bins with various levels ofimportance associated therewith;

FIG. 5 illustrates an advanced overwrite scheme to protect certain‘high-value’ video frames;

FIG. 6 further illustrates this overwrite scheme near the end of memoryspace;

FIG. 7 illustrates memory allocation with pointers to memory bins to besaved and pointers to those bins to be erased;

FIG. 8 illustrates four alternative timeline illustration in differentembodiments of the invention;

FIG. 9 is a block diagram of one embodiment of the invention;

FIG. 10 is a block diagram representing an embodiment of a methods formemory management; and

FIG. 11 is a more detailed block diagram of the method of FIG. 10.

DETAILED DESCRIPTION OF THE INVENTION

The present invention relates to apparatus and methods are provided foroverwriting memory in vehicle event recorder systems. Embodiments aredescribed hereinafter, that are constructed in accordance with theprinciples of the present invention, In order to facilitate theunderstanding of the described embodiments, definitions are provided forterms that may not be readily available in popular dictionaries.

Vehicle Event Recorders: Video image recording systems which areresponsive to triggers indicative of some event of interest.

Time Dilation: An expansion of a video sequence timeline by way of framerate manipulation.

Trigger: Electronic means for setting some instant in time associatedwith a particular event of interest and further for causing initiationof some associative processes.

Expanded Timeline Definition: A prescribed set of rules which sets forthand defines a timeline associated with a video frames sequence havingmore than one frame rate associated with any particular portion of thetimeline.

Overwrite Manager: A computer module determining, in accordance with anexpanded timeline definition, which data recorded in memory andassociated with a particular video frame is to be discarded and may beoverwritten with data from a newly collected video frame.

Video event recorder systems are typically built around and deployedwith memories of limited sizes, in order to contain cost. While massstorage and mass storage management may be included in such devices, forexample, a computer-type hard drive, these types of components remainsquite expensive, causing overall systems to double in cost if suchmemories were included. Instead, a ‘lightweight’ memory solution isenvisioned in the present invention, in which an abbreviated memory ormemory buffer is used to temporarily store information collected duringa predetermined time of service, for example a day, of a vehicleequipped with this type of video event recorder. Upon return to base, avehicle may transfer the information collected to a different memory formanagement and analysis. Accordingly, the present invention makes itpossible to equip vehicles with video event recorders having veryinexpensive cameras and memory.

Memories of such video event recording systems are preferably handled inthe following manner. A memory system is divided into two portions: afast, managed loop memory buffer, and a temporary mass storage memory.

Video continuously received from a video camera may be put into the fastmemory buffer. However, the amount of data generated by a video systemis quite extensive and most of the time totally uninteresting, butcertain portions of the video may become of great interest. For example,when a vehicle is involved in a traffic accident, the captured video mayyield important clues as to fault, cause, identity, and response, amongothers. In this event, it is important to preserve video data associatedwith these select video capture periods.

To this end, a trigger is arranged, whereby the occurrence of someincident of interest, such as an automobile accident, causes data storedin the memory buffer to be transferred to a more permanent memoryfacility. Old data in the memory buffer is continuously overwritten bynew data received from the video camera in real-time. In common andsimplistic versions, this step is performed in a “first-in,first-overwritten” manner.

Because the memory buffer is limited in its capacity to store videoframes, “first in first overwritten” schemes provide a timeline oflimited extent. For example, at a frame rate of four frames per second,a given memory buffer may be suitable for storing 30 seconds of videoframe data, or a 30 second video timeline. In ‘first-in, firstoverwritten’ schemes, this timeline may be arranged as 15 seconds ofcontinuous video before a trigger event and 15 seconds of continuousvideo after a trigger event.

However, a continuous frame rate throughout the entire event captureperiod need not be maintained. It is possible to have a modest framerate at times associated with the capture period extremities, and a highframe rate during periods around an event trigger. Therefore, thestorage frame rate may be adjusted throughout a prescribed capture timeperiod, allowing for an extended temporal range. Instead of a 30 secondtimeline, is entirely possible to have a 48 second timeline for the samememory. Such a timeline may be embodied as 12 seconds of video at aframe rate of one frame per second for the periods of time furthestapart from the event trigger, both before and after. In addition, thevideo sequence may include video for 24 continuous seconds, 12 secondsbefore and 12 seconds after an event trigger, at a video frame rate offour frames per second. This way, the temporal range is extended but thetemporal resolution is compromised in the time periods furthest from thetrigger event.

To create such a managed loop memory buffer management system, anoverwrite scheme is provided to select which frames are ‘expired’ and nolonger part of the particular extended timeline scheme. It should beremembered that video is continuously captured at all times, and thatvideo is captured at the maximum frame rate, because it is not known inadvance when an event trigger will occur. Accordingly, the system alwayscaptures video at the maximum frame rate as the capture frame ratecannot be adjusted in view of any event trigger which may come in thefuture.

Video captured at the maximum frame rate is put into the memory and asit is put in the memory it displaces previously recorded video frames.These frames are added to the memory locations determined to beavailable in accordance with a prescribed overwrite scheme such as theone mentioned above. However, this step is provided differently from afirst in first overwritten scheme. In the presently describedembodiment, most frames being overwritten are actually newer than atleast one other frame stored elsewhere in the memory buffer. Newlycaptured frames are written to memory positions in apseudo-‘interleaved’ fashion while some of older frames are preserved.

When a trigger event occurs, data in memory is transferred from thememory buffer to a memory of more permanent nature. When such data istransferred, an expanded timeline is reconstructed as a timeline havingat least two frame rates. At the extremities of the capture periodtimeline, the frame rate is reduced. At and about the point of greatestinterest (trigger event) the frame rate is maximized during the captureperiod. This ‘throttling’ of frame rate provides for a memory of presetsize to accommodate a timeline of greater temporal extent, although insome places resolution may be reduced.

Referring now to FIG. 1, a first timeline is shown that is associatedwith a memory system divided into a plurality of memory bins. Forexemplary purposes, some arbitrary numbers for memory size, number ofbins, video frame rates, etcetera, have been selected. It is to beunderstood that these are not necessarily preferred values, but valuesselected to promote an understanding of the example provided.

The memory related to FIG. 1 is a high-speed, high-performance memory oflimited extent, and is arranged as a buffer. This memory communicateswith incoming video data recorded by a video camera, and its output isdirected to another means for data storage means, such as a memorysystem having a greater capacity but lower speed, for example asemiconductor DRAM type memory. Alternatively, the memory of FIG. 1 maybe a non-volatile, high-performance memory based on ferromagneticprinciples, which can respond in real-time to video collected by a videocamera, but is of limited size and not suitable for saving the massamounts of data generated by video image systems.

In general, the memory of FIG. 1 may be limited to a few megabytes andmay temporarily hold a limited number of video frames, which may or maynot be transferred to a more permanent memory in a transfer operation.In particular, the memory may be divided into 120 bins, with each bin besufficient for storing the data associated with a single video frame.

A timeline 1 is associated timeline 1 with this memory, and is comprisedof a 30 second time interval. The timeline is marked in the Figure from0 to 30. A one second interval 2 is illustrated at the beginning of thetimeline. Further, that one second interval is divided into quadrants,representing a quarter of a second interval 3. For the video systems ofimmediate interest, his quarter of a second interval nicely accommodatesa single video frame (implicitly setting a frame rate of four frames persecond). While most modern video systems have far higher performancethan recording four frames per second, four frames per second is auseful rate for vehicle recorder systems, which tend to have limitedmemories in the interest of maintaining a low cost. Further, the kindsof events being recorded in vehicle recorder systems are appropriatelycaptured with frame rates of a few frames per second.

When video images are captured by a camera, frame-by-frame, each frameimages can be recorded into a memory bin 4. A first frame is recordedand put into a first memory bin. Thereafter, a quarter of second later,a second frame is recorded and put into an another memory bin, forexample, an adjacent bin. This frame-by-frame recording scheme maycontinue for up to 30 seconds before all memory bins becomes full andthe supply of empty bins is exhausted. In FIG. 1, the first 116 memorybins are shaded to indicate that one frame each of video data has beenwritten to those bins. This is equivalent to recording of a video signal5 of four frames per second for 29 seconds. FIG. 1 also illustrates fourempty memory bins 6, which would be filled in the next second of videorecording. Because the recording of video images in this manner is knownin the art, Figure a is labeled as prior art.

FIG. 2 illustrates a similar timeline 21 in conjunction with a graphicalillustration of a memory having 120 memory bins. As in FIG. 1, a timeinterval equivalent to one second 22 as well as a time interval of onequarter second 23 is illustrated for reference. The graphical depictionof the memory includes lightly shaded areas 24 and 25. The memory binspresented as 24 represent those bins having data written thereto from avideo which was collected from a time t=14 up to a time t=30. Thedemarcation indicated as dotted line 26 indicates time t=14. At timet=30, the memory is completely full. Video data collected for 30 secondsat four frames per second fills 120 memory bins. The video datacollected at time t=31 cannot be saved to memory unless a portion of thememory already allocated and consumed in a previous data write step isoverwritten. Thus, in the graphic of FIG. 2, memory bins indicated by 25on a second line represent-that video frame data that is recorded inthese memory bins at the expense of data captured 30 seconds prior.Accordingly, for the time period indicated, i.e. video data collectedfrom t=0 to t=14, that video data is lost to an overwrite step. In FIG.2, those bins shaded dark are indicated as 27, representing theover-written bins. This illustrates the so-called ‘round-robin’ or‘first-in, first-overwritten’ FIFO memory management schemes. Sincethese schemes are also known in the art, FIG. 2 is also labeled as priorart.

The FIFO memory management scheme is very useful. When a new video frameis collected by the video camera, it is placed into memory at the samelocation as the oldest frame in the memory which is discarded in theoverwrite step. Therefore, the FIFO memory management scheme impliesthat the oldest video information in the memory is the least valuable.

The memory described is a buffer memory, that is, this memorytemporarily holds the data of a video series for some specified time,but also continuously discards previously recorded information. When thebuffer contains a data set associated with an important event, that datais transferred from the buffer memory to a more permanent memory beforebecoming subject to being lost by overwrite actions. A video seriesbecomes ‘important’ when a detectable event occurs which implicitlyindicates video is valuable; for example, if a vehicle is involved in atraffic accident, accelerometers can detect the accident and trigger atransfer of data from the buffer memory to a permanent memory.

In those vehicle event recorder systems, a trigger is sometimes arrangedto indicate that such an event has occurred, that is, an event for whichthe video images associated therewith may be of extreme importance. Inthis case, the short term buffer memory of 120 video frames should betransferred to a more permanent long-term memory for example, a durableflash type memory.

FIG. 3 is directed at illustrating a timeline which includes an eventmoment. FIG. 3 includes a timeline 31, and the dashed line 32 toindicate the 29th second along with a marker ‘X’ 33 to indicate atrigger event has occurred at the 29th second. When a trigger eventoccurs, it is important to preserve the video data which occurred afterthe accident as well as the video data which occurred before theaccident. Video images collected during a time period starting 15seconds before the accident are in the bins indicated by 34; i.e. thosevideo image frames collected between t=14 and t=29. Memory bins at theend of the time line indicated by 35 include four video frames collectedduring the first second after the accident. Video image frames collectedbetween t=30 and t 44 are placed in the memory bins indicated by 36.Thus, the memory buffer contains video images for the 15 seconds priorto the accident and the seconds after the accident. Because the memoryis of limited size, it can only hold video image data which represents30 seconds of video recording.

At this point in time, no new frames are recorded to memory; overwriteis prevented, and the memory buffer is “locked”. Rather, the systempauses to transfer data in the buffer memory to a permanent flashmemory. After data is successfully transferred to the flash memory, thebuffer is “unlocked” and may be used again in the fashion described. Asvideo data which was placed into buffer memory bins between time t=30and time t=44, it caused older data to be displaced, overwritten andforever destroyed. Data which was recorded between t=0 and t=14 iscompletely lost and no access is possible any more to this information,which at one time resided in those memory bins, because that informationwas destroyed in the overwrite step. However, some of this informationmay be very valuable and, accordingly, it is quite undesirable to loseit entirely; in fact, some of this data may be more important than datawhich saved in its place.

Since the moments leading to a vehicle accident can explain a great dealabout the what actually happened, it is highly desirable to have atleast some limited information that relates to the accident scene att=1, for example. If one can just see one frame at t=1, that may beextremely valuable in explaining what happened in the accident.Therefore, the FIFO scheme may actually destroy critically useful data.

This is also apparent from FIG. 4, which explicitly shows certain binsA-F associated with various points of the timeline 41 and with referenceto trigger event 42 time at time t=29. The following discussion furtherillustrates the importance of bins A-F.

In a FIFO system, all memory bins, indicated by reference numerals 44and 45, are preserved in the memory buffer. Amongst the oldest recordedvideo frames remaining are those which reside in memory bins A and B,and which represent two adjacent frames, or frames captured within aquarter of a second from each other. Since these frames represent imagesvery close in time, these frames are expected to be quite similar toeach other. While it is sometimes desirable in video systems to havehigh temporal resolution, i.e. as many frames per second as possible,one will appreciate that at higher frame rates, a frame will containvery similar information as the frame closest thereto. Accordingly,where memory is limited, these adjacent frames lose their importance asmost of the information contained in each frame is similarly containedin the adjacent frame. Thus, if we keep frame A and discard frame B,most of the information of frame B can be known by examining frame A.

On the other hand, frames D, E and F, which are discarded in a FIFOsystem, may actually contain extremely important information. Frame D isseparated in time from frame E by one second. In a video scene, theremay be considerable differences between one frame captured an entiresecond later than another frame. Further, frame D occurs a full 29seconds before the trigger event. In a traffic accident, it can be quiteuseful to know about what was happening at time periods before and aftera trigger moment. Thus, it may be possible in a memory having a finitenumber of memory bins to trade some of the bins associated with lessimportant time slots for bins associated with time slots having agreater importance. If we discard frame B, and preserve frame D, we maygain a greater overall understanding of the incident being recorded. Ineffect, we can trade some time resolution (frame rate) at t=15, forimproved overall temporal range to realize an extended timeline.

One skilled in the art will notice that if video data associated with aframe rate of one frame per second was preserved, in seconds 1-12, then36 memory bins into would remain available, which would accommodatenewly captured video data. Thus, rather than completely overriding theoldest video data in memory, one can perform an overwrite action on 3 ofevery 4 memory bins in the overwrite portion of the timeline, therebymaintaining ¼^(th) of the oldest video data in those memory bins. Thatis to say, for the oldest video data in memory, it may be useful to saveone frame per second. To this end, when the overwrite operation isexecuted, new data is written to three memory bins, before one bin isskipped, and the process is repeated.

Timeline 51 includes a trigger event 52 at time t=29. In one overwritescheme of interest, it is required that a timeline be comprised of 12seconds of low temporal resolution, 24 seconds full temporal resolutionand a further 12 seconds of low temporal resolution. This is furtherdefined in detail as: a 12 second period of one frame per second video,a 24 second period of four frames per second video, and finally a 12second period of one frame per second; for a total video sequence of 48seconds. Since it cannot be known at what time in the future an eventtrigger will occur, a data overwrite scheme must preserve dataassociated with various frames, of which a prescribed timeline iscomprised. In the present example, continuous video data at a frame rateof four frames per second is preserved for a period of 12 seconds 54before the trigger event; that data is in memory bins indicated by 53.While in the FIFO system one can preserve data at four frames per secondfor up to 15 seconds before and after the trigger event, in the systemof the present embodiment only 12 seconds of four frames per second databe kept. However, it will be shown that the present embodiment enablesthe expansion of the total timeline of the video sequence to 48 secondsin contrast to the 30-second timeline of the FIFO system.

In the 31st second, the first overwrite operation begins. Whether or nota trigger has occurred, newly captured video data is written to everythree out of four memory bins, leaving the fourth memory binundisturbed. Therefore, old data is preserved, albeit at one quarter ofthe frame rate from which it was originally recorded. Video data afterthe trigger event is recorded in the memory bins 55 at a frame rate offour frames per second. Just because some bins are skipped, the framerate of video data collected after the trigger event is not necessarilyreduced. This is readily understood in consideration of the time pointindicated by 57 which indicates the time t=41 seconds, while, withoutskipping bins, this point in memory would have been time t=45. Carefulobservation will prove that the bins indicated by 55 will accommodatedata at four frames per second for the entire 12 seconds after the eventtrigger.

After the time point indicated by 57, several memory bins remainavailable for further overwrite operation before reaching the memorybins which contain data to be. preserved in agreement with the timelinedefinition 12/24/12. At least some of those memory bins up to theposition indicated by 54 are available for overwrite. After the full 12seconds of four frames per second video is recorded, it is desirable tocontinue recording video data at one frame per second for an additional12 seconds. Data captured in this period can be stored in memory bins,which are scattered in various locations about the memory buffer. FIG. 6illustrates on example of such locations.

More particularly, FIG. 6 illustrates memory bin locations which areavailable for overwrite as the memory approaches its full capacity forthe particular schemes presented herein. Once a trigger event occurs,i.e. is set in time, it is possible to compute which video frames mustbe saved in accordance with the particular timeline definition, andwhich frames may be discarded. For example, 48 frames at four frames persecond may be preserved immediately before the trigger event. Inaddition, 12 frames at a video rate of one frame per second may bepreserved for the time t=5 up to t=17. These frames must be protectedfrom any further overwrite operation, and are marked “must be saved” inFIG. 6. These frames are saved as they are included in the timelinedefinition.

All frames which precede t=5 are in condition for being discarded. thatis, such frame lie outside the time range which is to be preserved.Accordingly, frames indicated for example as 69 have aged sufficientlyand are may be erased. These are the frames which originally werepreserved in the overwrite operation as skipped frames.

Video frames captured after the trigger event are also saved in thememory. For 12 seconds after the trigger event, t=29 to t=41, video iscaptured at a rate of four frames per second. Such a video data 65 isput into memory in accordance with the need to save particular frames ofthe oldest video data. When all video frames from the period t=29 tot=41 are properly recorded, the system continues to record data at theframe rate of one frame per second. This is different from the earlieroperation, in which the overwrite action resulted in the preservation ofone frame per second.

For the time period 12 seconds after the event trigger up to 24 secondsafter the event trigger, data is put into memory at the reduced framerate of one frame per second. Other frames may be captured by thecamera, but are discarded before entering the memory or instantlythereupon. Thus, the frames represented by 67 are put into memory binswhich are available in accordance with the “OK to erase” label in thedrawing. A person skilled in the art will note that after three of theseframes are placed in the memory, the fourth frame 68 cannot be placedinto the memory in the same repeating geometric position. That is tosay, those memory bins are not available for overwrite. Therefore, videocaptured after that time must be carefully managed and fit into theavailable memory bins.

FIG. 7 illustrates the steps taken in the final filling of the remainingmemory bins. In timeline 71, event trigger 72 is situated at time t=29.In agreement with this exemplary timeline definition, video captured ata frame rate of four frames per second from t=17 to t=29 is stored inmemory, as indicated by 73. Similarly, video captured for a 12 secondperiod at a frame rate of four frames per second from t=29 to t=41 isstored in memory, as shown by reference numeral 74. Finally, videoframes captured during a 12 second period from t=42 to t=54 at a framerate of one frame per second include those particular frames representedas 75, which must be inserted into the memory bins remaining availablefor overwrite.

Arrows 76 indicate that these frames may be placed in locations near thebeginning of the memory, where data had once been stored but is nowexpired because the trigger event occurs at t=29. Once a trigger eventis established, the bins which may be overwritten can be determinedaccording to the particular rules defining the timeline.

The example of FIG. 7 clearly illustrates that careful management of anoverwrite scheme enables a memory buffer to dilate a timeline bymanipulating which video frames are preserved and which are overwritten.Consequently, temporal resolution is sacrificed to extend temporalrange, that is, the frame rate of “saved data” is altered in order tomake more space available for video frames captured further in time fromthe event trigger. Accordingly, the greatest amount of information canbe preserved in a memory buffer of the limited size.

While the example of FIG. 7 illustrates where the data may be written inmemory, those skilled in the art will note that the physical positionsof memory bins may be altered. Therefore, after a timeline definition isset, an algorithm may be developed defining the bins containing datathat has expired and thus implicitly defining a bin available foroverwrite at any moment time.

While the example presented of FIGS. 5-7 illustrates one possiblesolution, it should be understood that other arrangements may providefor a time dilation in accordance with the spirit of the presentinvention, and that specific values may be used that are different fromthose presented in the above exemplary timeline definition. In anotherexemplary timeline definition, one might arrange a system whereby twoperiods of eight seconds are used to capture video of a high frame rate,and two periods of 28 seconds are used to capture data at a low framerate, thus achieving a total expanded timeline of 72 seconds.

The advantages offered by the above examples do not depend upon theparticular values chosen in these examples. One should also recognizethat because capturing/saving video at two different frame rates enablesa user one to expand the timeline, capturing/saving video at threedifferent frame rates also enables a user to expand the timeline withgreater flexibility. Accordingly, the memory may be manages to preserveframes for some time periods at a rate of four frames per second, and inother time periods at a rate of two frames per second, and in stillother time periods at a rate of one frame per second. This arrangementprovides for very high temporal resolutions for the periods immediatelysurrounding an accident (trigger event), for medium level resolutionsfor periods further away from the trigger event, and finally for lowtemporal resolutions at the extremities of the time range.

In addition, asymmetric timeline definitions are possible, that is, thetime periods on either side of the event trigger may not be equal inextent or in number. A timeline definition may be devised that has along, high resolution period before the event trigger, and a short highresolution period after the event trigger. FIG. 8 illustrates varioustimeline definitions of interest, and is related to several exampleseach working equally well within the common concept of timelinedilation. FIG. 8 graphically illustrates a first memory buffer 81, whichwas discussed in detail in a previous example, and in which there aretwo frame rates, namely, a high video frame rate of four frames persecond and a low video frame rate of one frame per second.

A trigger event 82 occurring at some instant in time implicitly sets thetime periods for any particular example, and time period 83 startsimmediately after the trigger event and extends for 12 seconds. A secondtime period 84 extends from the trigger event to 12 seconds prior to thetrigger event. In both of these time periods, video is captured and putinto the memory buffer at a rate of four frames per second.

The number of shaded memory bins reflects a frame rate of 4 frames persecond. Time periods at the extremities of the timeline, periods 85 and86, are each also configured to be 12 seconds in length. However, sinceonly one frame per second is collected in those time periods, the numberof memory bins consumed is considerably smaller, i.e. ¼ of thoseconsumed in the other time periods. This arrangement provides for atotal timeline of 48 seconds, and in memory buffers that do notoverwrite/store data at variable rates, the same memory size could onlyaccommodate a timeline of 30 seconds. FIFO memories of the same size arerestricted to 30 seconds.

A second example presented as 87 in FIG. 8 suggests two high temporalresolution periods of 10 seconds each. In addition, there are two lowtemporal resolution periods of 20 seconds each. While there is a reducedoverall period of high-resolution video data, the total timeline isextended to 60 seconds.

A third example is presented through the memory buffer of graphic 88,and illustrates that an asymmetric timeline definition may also beconfigured. The two periods with a high rate of video recording need notbe the same in extent. In fact, video may be recorded at a high framerate for a longer period after a trigger event than that in the periodimmediately preceding the trigger event. In the present example, videois recorded in the memory buffer for 16 seconds after the trigger event,but only for four seconds prior to the trigger event. Accordingly, thetotal high-resolution time period is the same as in the previousexample, 20 seconds, but greatly favors preserving information after thetrigger event, at the expense of information preceding the triggerevent.

In a fourth example, there are six distinct time periods comprised inthe timeline. Two 9 second periods occur symmetrically about an eventtrigger. In these time periods video may be captured a rate of fourframes per second. Two additional periods each of 8 seconds may be usedto record/overwrite data at a frame rate of two frames per second. Twoadditional 8 second periods are provided to store data at a frame rateof one frame per second. One skilled in the art will appreciate that inthe timeline of this example, two of the 8 second periods are ofdifferent sizes with respect to memory capacity, i.e. greater number ofbins, than the other two 8 second periods. This is consistent with thehigher frame rate used in two of the 8 second periods.

One skilled in the art will also appreciate the great latitude availablefor managing a memory buffer of limited capacity to expand a timeline.One skilled in the art will further appreciates that where memorybuffers deploy FIFO or ‘round-robin’ strategies for overwriteoperations, very important data may be lost. FIFO and ‘round-robin’strategies discriminate against the oldest data in a memory buffer, andin situations where the oldest data is not the least valuable, FIFO andround-robin systems are inferior to the system of the present invention.

Referring now to FIG. 9, the fundamental elements of apparatus accordingto the present invention is described. Video camera 91 is operable forcollecting optical energy and for converting the image of a scene intoelectrical signals, suitable for processing by common electronic meanssuch as digital semiconductor memories and processors. In addition,these systems include a trigger mechanism 92.

In one embodiment, a trigger mechanism is the device arranged to providean electrical signal that indo indicates that a particular video seriesshould be transferred to permanent memory for long-term storage. Atrigger may be an accelerometer operable for detecting abrupt changes inspeed, for example, speed changes related to a traffic accident.Triggers may be activated by other events such as heavy braking orswerving maneuvers, and may be activated by means other thanaccelerometers. For example, a user panic button can be used to activatea trigger event.

When the user believes that a video series should be saved, he can hit apanic button to activate one type of trigger. It is not relevant whatprecisely causes a trigger to be activated, but rather how memoryperforms once a trigger event has occurred. Overwrite manager 93 is acontrol module that interfaces with the trigger and a video camera, andalso with a buffer memory 94. An overwrite manager includes means wherea timeline definition may be set and further means for executingoverwrite operations in agreement with the stored timeline definitions.Further, an overwrite manager may additionally integrate withflush-module 95.

When a trigger event occurs, overwrite manager 93 continues to overwritedata to buffer memory 94 in accordance with the timeline definition, byway of an overwrite pointer which is associated with a cell subject toan impending overwrite action. Overwrite manager 93 sends a signal 96 toflush module 95 that cause flush module 95 to copy buffer memory 94 andto transfer the video data set with the prescribed expanded timeline tohigh-capacity long-term storage 97. Overwrite manager 93 controls thealgorithms and the necessary processing components for writing to buffermemory 94 and save selected data while purging redundant data inaccordance with a particular expanded timeline definition.

FIGS. 10 and 11 which illustrate the primary steps of methods inaccordance with the present invention. In particular, FIG. 10 describesuch methods in the most general sense to include step 101, wherebyframe data is received from a video camera, and step 102, whereby thenewly received data is written over old data stored in the memory bufferaccording to an expanded timeline definition.

FIG. 11 illustrates these methods in greater detail. Frame data 111 isreceived from a video camera in a first step. Buffer memory data writestep 113 includes sub-step 114, in which the frame is written to a binmarked open. It is important that data be written in the buffer memoryin an organized fashion, without disturbing particular data frames,necessary to fill the prescribed expanded timeline definition.Therefore, a bin is marked ‘open’ when it no longer contains frame datanecessary for the expanded timeline definition.

In second sub-step 115, a determination is made as to which memory bincontains frame data that is no longer needed in agreement with thetimeline definition. This determination made during each cycle. Forevery new frame entering the buffer memory, another frame becomes nolonger necessary at the same instant.

Finally, in third sub-step 116, the bin which contained data that is nolonger required is marked ‘open’. In following cycle 112, the nextincoming frame is written to the appropriate bin. It is helpful to set abuffer memory pointer to direct the incoming frame to a bin marked‘open’.

One skilled in the art will appreciate that advanced memory managementschemes may be deployed to expand a recorded timeline in memory buffershaving limited capacity.

While embodiments of the invention have been described above, it will beapparent to one skilled in the art that various changes andmodifications may be made. The appended claims are intended to cover allsuch changes and modifications that fall within the true spirit andscope of the invention.

What is claimed is:
 1. A vehicle event recorder configured to be mountedin a vehicle, the vehicle event recorder comprising: a video camerahaving a field-of-view that includes an environment about the vehicle,the video camera configured to acquire consecutive frames of visualinformation representing the environment about the vehicle; a buffermemory configured to electronically store information; one or moresensors configured to generate output signals conveying informationrelated to vehicle events; and a controller configured to: effectuatestorage of the consecutive frames of visual information in the buffermemory with a first frame storage density; detect vehicle events basedon the output signals; and responsive to detecting a vehicle event,designate individual frames of visual information stored in the buffermemory related to the vehicle event for long-term storage, wherein theindividual frames related to the vehicle event designated for long-termstorage have the first frame storage density during a first period oftime that includes the vehicle event, wherein the individual framesrelated to the vehicle event identified for long-term storage have asecond frame storage density for a second period of time before thefirst period of time and a third period of time after the first periodof time, wherein the first frame storage density is higher than thesecond frame storage density, and wherein the controller is configuredto facilitate interleaved overwriting of frames in the buffer memory notrelated to the vehicle event and not designated for long-term storagewith additional frames acquired by the video camera after detection ofthe vehicle event, and wherein at least one of the additional framesacquired by the video camera after detection of the vehicle event isdesignated for long-term storage.
 2. The vehicle event recorder of claim1, wherein the controller is configured such that the individual framesdesignated for long-term storage have a variable frame storage densitythat corresponds to an amount of time from the vehicle event on avehicle event timeline such that the individual frames designated forlong-term storage are more dense nearer in time to the vehicle event andless dense farther in time from the vehicle event.
 3. The vehicle eventrecorder of claim 1, wherein the controller is configured such that thefirst frame storage density and the second frame storage density arerelated to a number of frames stored per a number of consecutive framesacquired by the video camera.
 4. The vehicle event recorder of claim 1,wherein the controller is configured such that the first period of time,the second period of time and the third period of time are the same. 5.The vehicle event recorder of claim 1, wherein the controller isconfigured such that the second period of time and the third period oftime are different.
 6. The vehicle event recorder of claim 1, whereinthe controller is configured such that individual frames designated forlong-term storage during the third period of time have a third framestorage density that is different than the first frame storage densityand the second frame storage density.
 7. The vehicle event recorder ofclaim 1, wherein the one or more sensors include an accelerometer. 8.The vehicle event recorder of claim 1, further comprising a long-termmemory configured to electronically store information, wherein thecontroller is configured to facilitate transferring the individualframes designated for long-term storage from the buffer memory to thelong-term memory.
 9. The vehicle event recorder of claim 8, wherein thecontroller is configured to facilitate transfer of the individual framesdesignated for long-term storage responsive to the buffer memory beingfilled to capacity with frames designated for long-term storage.
 10. Thevehicle event recorder of claim 8, wherein, responsive to the transfer,the controller resets the buffer memory to an initial state wherein noindividual frames are designated for long-term storage.
 11. The vehicleevent recorder of claim 8, wherein the long-term memory is configured tostore individual frames related to a plurality of vehicle events.
 12. Amethod for storing visual information related to a detected vehicleevent, the method comprising: acquiring consecutive frames of visualinformation representing the environment about a vehicle with a videocamera having a field-of-view that includes an environment about thevehicle; effectuating storage of the consecutive frames of visualinformation in a buffer memory with a first frame storage density;generating output signals conveying information related to vehicleevents; detecting vehicle events based on the output signals; andresponsive to detecting a vehicle event, designating individual framesof visual information stored in the buffer memory related to the vehicleevent for long-term storage, wherein the individual frames related tothe vehicle event designated for long-term storage have the first framestorage density during a first period of time that includes the vehicleevent, wherein the individual frames related to the vehicle eventidentified for long-term storage have a second frame storage density fora second period of time before the first period of time and a thirdperiod of time after the first period of time, and wherein the firstframe storage density is higher than the second frame storage density,and facilitating interleaved overwriting of frames in the buffer memorynot related to the vehicle event and not designated for long-termstorage with additional frames acquired by the video camera afterdetection of the vehicle event, and wherein at least one of theadditional frames acquired by the video camera after detection of thevehicle event is designated for long-term storage.